THE ENTROPY OF RANDOM-FREE GRAPHONS AND PROPERTIES 



HAMED HATAMI AND SERGUEI NORINE 



Abstract. Every graphon defines a random graph on any given number n of vertices. It was known that 
the graphon is random-free if and only if the entropy of this random graph is subquadratic. We prove 
that for random-free graphons, this entropy can grow as fast as any subquadratic function. However, if the 
graphon belongs to the closure of a random-free graph property, then the entropy is 0(n log n). We also give 
a simple construction of a non-stepfunction random-free graphon for which this entropy is linear, refuting a 
conjecture of Janson. 



1. Introduction 

In recent years a theory of convergent sequences of dense graphs has been developed. One can construct a 
limit object for such a sequence in the form of certain symmetric measurable functions called graphons. Every 
graphon defines a random graph on any given number of vertices. In [HJS] several facts about the asymptotics 
of the entropies of these random variables are established. These results provide a good understanding of 
the situation when the graphon is not "random-free". However in the case of the random-free graphons they 
completely trivialize. The purpose of this article is to study these entropies in the case of the random-free 
graphons. 

1.1. Preliminaries. For every natural number n, denote [n] := {1, ...,n}. In this paper all graphs are 
simple and finite. For a graph G, let V(G) and E(G), respectively denote the set of the vertices and the 
edges of G. Let U denote set of all graphs up to an isomorphism. Moreover, for n > 0, let U n C U denote 
the set of all graphs in IA with exactly n vertices. We will usually work with labeled graphs. For every n > 1, 
denote by C n the set of all graphs with vertex set [n] . 

The homomorphism density of a graph H in a graph G, denoted by t(H;G), is the probability that a 
random mapping <fi : V(H) — > V(G) preserves adjacencies, i.e. uv e E{H) implies <f>(u)</)(v) £ E(G). The 
induced density of a graph H in a graph G, denoted by p(H ; G), is the probability that a random embedding 
of the vertices of H in the vertices of G is an embedding of H in G. 

We call a sequence of finite graphs (G n )^L 1 convergent if for every finite graph H , the sequence {p(H; G n )} 
converges. It is not difficult to construct convergent sequences (G n )^ =1 such that their limits cannot be rec- 
ognized as graphs, i.e. there is no graph G, with lim n _ ! . 0O p(H ; G n ) = p(H; G) for every H . Thus naturally 
one considers U, the completion of U under this notion of convergence. It is not hard to see that U is a 
compact metrizable space which contains U as a dense subset. The elements of the complement U°° :=U\U 
are called graph limits. Note that a sequence of graphs (G„)™ =1 converges to a graph limit T if and only if 
| V(G„)| — > oo and p(H; G„) — > p(H; T) for every graph H . Moreover, a graph limit is uniquely determined 
by the numbers p(H; V) for all H eW. 

It is shown in [LS06] that every graph limit T can be represented by a graphon, which is a symmetric 
measurable function W : [0, l] 2 — >• [0,1]. The set of all graphons are denoted by Wq. Given a graph G 
with vertex set [n], we define the corresponding graphon Wq '■ [0, l] 2 — > {0, 1} as follows. Let Wo{x,y) := 
j4g(M, \yn\) if x,y £ (0, 1], and if x — or y = 0, set Wg to 0. It is easy to see that if (G n )^ =1 is a graph 
sequence that converges to a graph limit T, then for every graph H , 
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p(H; T) = lim E 



) (1- W Gn (x u ,x v )) 

uv£E{H) uv&E{H)" 



where {x u }uev(H) are independent random variables taking values in [0,1] uniformly, and E(H) C = {uv : 
u ^ v, uv ^ E(H)}. Lovasz and Szegedy |LS06| showed that for every graph limit L, there exists a graphon 
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W such that for every graph H , we have p(H; T) = p(H; W) where 



p(H; W) := E 

Furthermore, this graphon is unique in the following sense: If Wi and W 2 are two different graphons repre- 
senting the same graph limit, then there exists a measure-preserving map a : [0, 1] — >• [0, 1] such that 

W 1 (x,y) = W 2 (a(x),a(y)), (1.1) 

almost everywhere [BCLlOj . With these considerations, sometimes we shall not distinguish between the 
graph limits and their corresponding graphons. We define the Si distance of two graphons W\ and W 2 as 

h{Wi,W 2 ) = inf || Wi - W 2 o tr||i, 

where the infimum is over all measure-preserving maps a : [0, 1] — > [0, 1]. 

A graphon W is called a stepfunction, if there is a partition of [0, 1] into a finite number of measurable 
sets Si, ■ . . ,S„ so that W is constant on every Si x Sj. The partition classes will be called the steps of W. 

Let W be a graphon and x\, . . . ,x n G [0, 1]. The random graph G(xi, . . . , x n , W) G C n is obtained by 
including the edge ij with probability W(xi,Xj), independently for all pairs with 1 < i < j < n. 

By picking xi,...,x n independently and uniformly at random from [0,1], we obtain the random graph 
G(n, W) G C n . Note that that for every H G C n , 

Pr[G(n, = H] = p(H; W). 

1.2. Graph properties and Entropy. A subset of the set U is called a graph class. Similarly a graph 
property is a property of graphs that is invariant under graph isomorphisms. There is an obvious one-to-one 
correspondence between graph classes and graph properties and we will not distinguish between a graph 
property and the corresponding class. Let Q C U be a graph class. For every n > 1, we denote by Q n the 
set of graphs in Q with exactly n vertices. We let QCM be the closure of Q in U. 

Define the binary entropy function h : [0, 1] H> R+ as h(x) = —x log(a;) — (1 — x) log(l — x) for x G (0, 1) 
and h(Q) = h(l) = so that h is continuous on [0, 1] where here and throughout the paper log(-) denotes 
the logarithm to the base 2. The entropy of a graphon W is defined as 

Ent(W) := f [ h(W(x,y))dxdy. 
Jo Jo 

Note that it follows from the uniqueness result (jl.ljl that entropy is a function of the underling graph limit, 
and it does not depend on the choice of the graphon representing it. It is shown in |Ald85| and [Jan, Theorem 
D.5] that 

,MMM = E „ tW . (1 . 2) 

(a) 

A graphon is called random-free if it is {0, l}-valued almost everywhere. Note that a graphon W is random- 
free if and only if Ent(VK) = 0, which by (|1.2p is equivalent to Ent(G(n, W)) = o(n 2 ). Our first theorem 
shows that this is sharp in the sense that the growth of Ent(G(n, W)) for random-free graphons W can be 
arbitrarily close to quadratic. 

Theorem 1.1. Let a : N — > be a function with linin^oo a(n) = 0. Then there exists a random-free 
graphon W such that Ent(G(n, W)) = fl(a(n)n 2 ). 

A graph property Q is called random-free if every W G Q is random-free. Our next theorem shows that 
in contrast to Theorem 11.11 when a graphon W is the limit of a sequence of graphs with a random-free 
property, then Ent(G(n, W)) cannot grow faster than 0{n\ogn). 

Theorem 1.2. Let Q be a random-free property, and let W be the limit of a sequence of graphs in Q. Then 
Ent(G(n,W)) = 0(n log n). 
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Remark 1.3. We defined G(n,W) as a labeled graph in C n . Both Theorems 11.11 and 11.21 remain valid if we 
consider the random variable G u (n, W) taking values in U n obtained from G(n, W) by forgetting the labels. 
Indeed, Ent(G u (n, W)) = Ent(G(n,W)) - Ent(G(n,W) \ G u {n,W)) and Ent(G(n,W) \ G u (n,W) = H) = 
0(n\ogn) for every H £ U n . It follows that 

Ent(G(n,W)) -0(n log n) < Ent(G„(n, W)) < Ent(G(n, W)). 



2. Proof of Theorem 11.11 

For every positive integer m, let F m denote the unique bigraph ([m], [2 m },E) with the property that the 
vertices in [2 m ] all have different sets of neighbors. The transversal-uniform graph is the unique graph (up 
to an isomorphism) with vertex set N which satisfies the following property. The vertices are partitioned 
into sets {Aj}?^ with log \Ai\ = X^'=i l^i-il- There are no edges inside A^s, and for every i, the bigraph 
induced by (U'Z^A,-, Ai) is isomorphic to F^- 1 \a-\- 

Let X = {Ii}i£jq be a partition of [0, 1] into intervals. We define its corresponding transversal-uniform 
graphon Wx by assigning weights to all the vertices in Ai in the transversal- uniform graph Gjj 

described above. More precisely, we partition each Ii into \Ai\ equal size intervals (corresponded with 
elements in Ai), and mapping all the points in each of these subintervals to its corresponding vertex in Ai. 
This measurable surjection irx : [0, 1] — > N, together with the transversal-uniform graph described above 
defines the transversal-uniform graphon Wx by setting 



Wx(x,y) 



1 i£ir(xMy)eE(Gu), 
if7r(x)7r(y) #E(Gu). 



Note that by construction Wx has the following property. Let s < k be positive integers, and X\,...,x s € 
^i<kli belong to pairwise distinct intervals in X. For every / : [s] — > {0, 1}, we have 

Pr[V i; Wx(xi,y) = f(i)\yel k } = ±- s , 
where y is a random variable taking values uniformly in [0, 1]. It follows that for every graph H on s vertices, 

Pr[G(a;i,...,x.,W r z)=fl"|Vi ) x i eI ki ] = -Lr, (2.1) 

where x\,...,x s are now i.i.d. random variables taking values uniformly in [0,1], and k\, k%, . . . , k s are 
distinct natural numbers. 

We translate (j2.1j) into a lower bound on (conditional) entropy of transversal-uniform graphons. First we 
need a simple lemma. 

Lemma 2.1. Let Wx be a transversal-uniform graphon, and <j) : [n] — > [0,1] be a uniformly random map. 
For every p : [n] — > N, we have 



Ent(G(0(l), . . . , cj>{n),Wx) \ir x ° = p) > 



/|Im(p)|\ 



Proof. Pick a set of representatives K C [n] so that p\x ■ K — > Im(p) is a bijection. Equation (|2.1|) implies 
that for every graph H with V(H) = K, 

Pr[G(<Xl), ■ • ■ , <f>(n),Wx)[K\ = H \ nx o = p] = 



2i 2 j 



Therefore, 



Ent(G(0(l), . . . , 4>{n),Wx) | ttx o - p) > Ent(G(0(l), . . . , c/>(n),Wx)[K) \ vn o c/> = p) = (^ lm ^ . 



□ 



In the proof of Theorem 11.11 below we will make use of the following well-known inequality about condi- 
tional entropy. For discrete random variables X and Y, 

Ent(X \ Y):= ^ Pr[Y = y]Ent(A \Y = y) < Ent(A). (2.2) 

yesupp(r) 

Proof of Theorem ] 1. 11 For every positive integer k, define 

g k :=max{{2 fc+5 }U{n|a(n) > 2- 2k ~ 9 }}. 

The numbers g k are well-defined, as the condition linin^oo a(n) = implies that the set {n \ a(n) > 2~ 2k ~ 9 } 
is finite. Define the sums G k := X)i=ifffc; an d se ^ Pi = —^F ^ or au * ne # fe indices i <E (Gk-i,Gk\- Let 
I = {Jj}jgN be a partition of [0, 1] into intervals with \Ii \ = (3i, and let Wx be the corresponding transversal- 
uniform graphon. 

Consider a sufficiently large n G N, and let k £ N be chosen to be maximum so that 2 k+A < n and 
a(n) < 2~ 2k ~ 7 . We have n < 2 k+5 or a{n) > 2~ 2k ~ 9 . Therefore n < g k by the definition of g k . Let 
4> : [n] — > [0, 1] be random and uniform. By Lemma |2. 11 for any fixed p : [n] — > N, we have 

Ent(G(0(l), . . . , <P{n),Wx)\nx ° = p) > . 

Thus 

<n2- k ~ 2 



Ent(G(n,Wx)) > Ent(G(n, Wx)|ttx o 0) > Pr QlnuVx o 0)| > n2~ k ~ 2 ] I ^ ). (2.3) 
Define the random variable X := |Im(7ri o <f>) n (Gk-i, Gk)\ < Im(7rx o </>)|. We have 



l[X]= Yl P^~ 1 (h)^9]= Yl (i-C 1 - &)") = 9k (l - (l - ~V) ") >n2" 



k-1 



where we used the fact that g k 2 k > 2n and that (1 — x) n < 1 — nx + n 2 x 2 < 1 — nx/2 for x € [0, l/2n]. As 
the events (/> _1 (/j) ^ and </> _1 (7j) ^ are negatively correlated for i ^ j, we have Var[X] < ELY]. Hence 
by Chebyshev's inequality 



Pr [\Im(xx °(f>)\> n2- k - 2 ] > Pr [X > n2~ k ~ 2 ] > 1 - Pr 



|X-E[X]|>1E 



> 4Var[A] 4 > 1 

" ELY] 2 " ~ n2- fc - 2 " 2' 



Substituting in ()2.3|) we obtain 



Ent(G(n, W x )) > I * ^ > n 2 2- 2fc - 7 > a(n)n 2 , 
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as desired. □ 



3. Proof of Theorem 11.21 

In |LS] Lovasz and Szegedy obtained a combinatorial characterization of random-free graph properties. 
To state this result it is convenient to distinguish between bipartite graphs and bigraphs. A bipartite graph 
is a graph (V, E) whose node set has a partition into two classes such that all edges connect nodes in different 
classes. A bigraph is a triple (Ui, U2,E) where U\ and Ui are finite sets and E C U\ x C/ 2 . So a bipartite 
graph becomes a bigraph if we fix a bipartition and specify which bipartition class is first and second. On the 
other hand, if F = (V, E) is a graph, then (V, V, E') is an associated bigraph, where E' = {(x, y) : xy G E}. 

If G = (V, E) is a graph, then an induced sub-bigraph of G is determined by two (not necessarily disjoin) 
subsets S, T C V, and its edge set consists of those pairs (x,y) € S xT for which xy € E (so this is an 
induced subgraph of the bigraph associated with G). 
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For a bigraph H = [U\, U2, E) and a graphon W, analogous to the definition of the induced density of a 
graph in a graphon, we define 



p b (H; W) = E 



W(x u ,y v ) (1-W(x u ,y v )) 



uv£E 



ueu ly v£U 2 

uve(U!XU 2 )\E 



where {x u } u gc/i , {yv}v£U 2 are independent random variables taking values in [0,1] uniformly. Now we are 
ready to state Lovasz and Szegedy's characterization of random-free graph properties. 



Theorem 3.1. |LS) A graph property Q is random-free if and only if there exists a bigraph H such that 
p h {H;W) = for all W G Q. 

The following lemma is due to Alon, Fischer, and Newman (See |AFN071 Lemma 1.6]). 

Lemma 3.2. [AFN07] Let k be a fixed integer and let 8 > be a small real. For every graph G, either there 

exists stepfunction graphon W' with r < steps such that 8\{Wg, W) < 8, or for every bigraph H on 

k vertices p b (H:G) > (f) ^ '. 

Every random-free graphon W can be approximated arbitrarily well in the 8\ distance with Wg for some 
graph G, and furthermore, for every fixed H, the function p b (H, •) is continuous in the 8\ distance. Thus 
Lemma 13.21 can be generalized to random- free graphons. 



Corollary 3.3. Let k be a fixed integer and let 5 > be a small real. For every random-free graphon W , 
either there exists a stepfunction graphon W' with r < (f)° ' steps such that 8i(W, W') < 8, or for every 
bigraph H on k vertices p b (H;G) > (r)° * ■ 

Next we will prove two simple lemmas about entropy. 
Lemma 3.4. Let [i\ and 112 be two discrete probabilistic distributions on a finite set f2. Then 

I Mi - H2W1" 



|Ent(/xi) - Ent(/i 2 ) < \Q\h 



\n\ 



Proof. Define OlogO := lim^^o-^log^ = 0. By taking the derivative with respect to x, for fixed d we see 
that (x + d) \og(x + d) — xlogx is monotone for < x < 1 — d. Therefore, for xi,X2 G [0, 1] we have 

\x2 log x 2 - x\ log a; 1 1 < max{— \x% - x\\ log \xi — x\\, — (1 — \x 2 - xi\) log(l - \x 2 - xi\)} < h(\x 2 - Xi\). 

Thus 



|Ent(Mi)-Ent(/x 2 )| = 



^2 ^ ( X ) l0g ( x ) - M2 (x) log ^2 (x) 



< ^2hi\in(x)-ii2(x)\)<\Q\h 



[Ml - Mgjli 
101 



□ 



where the last inequality is by concavity of the binary entropy function h. 

Lemma 3.5. Let W\ and W2 be two graphons, and let fix and 112 be the probability distributions on C n 
induced by G{n,W\) and G{n,W2), respectively. Then 

||A*i-A*2||i<n%(Wi,Wa). 

Proof. Let x\, . . . , x n be i.i.d. uniform random variables with values in [0, 1]. Note 

H/U1-M2II1 < Pr[G(xi,. . . ,x n ,Wi) ^ G(x 1 ,...,x n ,W 2 )] 



< E 



^2 \Wi(xi,Xj) - W 2 (xi,Xj)\ 

i±3 



<^||Wl-W2||l. 



□ 



Proof of Theorem \1.2l Since Q is random- free, by Theorem l3.ll there exists a bigraph H such that p b (H; W) = 
for all W 6 Q. Applying Corollary 13.31 with S = 1/ra 5 shows that there exists a stepfunction graphon W' 
with n°W steps satisfying \\W - W'\\\ < 5. Then since \C n \ < 2 n \ Lemmas EOl and |3~51 imply 



|Ent(G(n,W)) - Ent(G(n,W))\ < T h 



no 

9n 2 



"2" t^t log + log 1 



9n J ° y 2 n / y 2 / V 2 n 

< n 4 S + n 2 <5(-21ogn - log<5) + 2" • 2— = o(l). 

2™ 

Since W' is random-free and it has n°W steps, |supp(G(n, W'))\ = n°^ n \ Consequently Ent(G(ra, W')) = 
0{n log n). □ 



4. Concluding remarks 

1. Note that if W is a random-free stepfunction, then Ent(G(n, W)) = 0(n). In |Jan| it is conjectured that 
the converse is also true. That is Ent(G(n, W)) — 0(n) if and only if W is equivalent to a random-free 
stepfunction. The following simple example disproves this conjecture. 

Let /j, be the probability distribution on N defined by — 2~\ Consider the random variable 

X = (Xi, . . . , X n ) £ N™ where X{ are i.i.d. random variables with distribution /i. We have Ent(Xj) = 
T,Zi 2 ~ H = 2 - Hence Ent(X) < X)Ent(Xi) = In. 

Partition [0,1] into intervals {Ii}°Z 1 where = 2 _i . Let W be the graphon that is constant 1 on 
Ug.jJ, x Ii and everywhere else. Note that 

Ent(G(n, W)) < Ent(X) < 2n. 

Therefore G(n, W) has linear entropy. 

It remains to verify that W is not equivalent to a stepfunction. This follows immediately from the fact 
that W has infinite rank as a kernel. It can also be verified in a more combinatorial way: A homogenous 
set of vertices in a graph H is a set of vertices which are either all pairwise adjacent to each other, or all 
pairwise non-adjacent. If W is equivalent to a step-function with k steps, then every H £ supp(G(n, W)) 
cleary contains a homogenous set of size at least n/k. On the other hand, if H £ L n i is a disjoint union of 
n complete graphs on n vertices, then the largest homogenous set in H has size n, but H € supp(G(n 2 , W)) 
by construction. 

2. Theorem 11.21 shows that when W is a limit of a random-free property, then the entropy of G(n, W) is 
small. However, the support of G(n, W) can be comparatively large. For every e > 0, we construct examples 
for which log(|supp(G(n, W) ) | ) = fl(n 2 ~ e ). Note that Theorem 11.21 implies that G(n, W) is far from being 
uniform on the support in these examples, as the entropy of a uniform random variable with support of size 
2 n (" 2_e ) is VL{n 2 - e ). 

Let us now describe the construction. Let Q be the set of graphs that do not contain K t} t as a subgraph. 
Partition [0, 1] into intervals {Si}°°^ 1 with non-zero lengths, and let {Hi}°l 1 be an enumeration of graphs in 
Q. Define W to be the graphon that is on Si x Sj for i ^ j, and is equivalent to Wjii (scaled properly) on 
Si x Si. By construction p(H; W) > if H £ Q. Thus |supp(G(n, W))\ > |Q„|. Since there exists -ftT M -free 
graphs with n 2 " 2 /* edges (See e.g. |Bol781 p. 316, Thm. VI.2.10]), we have \Q n \ > 2" 2 ~ 2/ \ 

It remains to show that W is a limit of graphs in some random-free property. Unfortunately, W £ Q. We 
construct a larger random- free property Q' so that W £ Q' as follows. 

Fix a bigraph B, so that the corresponding graph contains K t ^ as a subgraph and is connected. Suppose 
further that no two vertices of B have the same neighborhood. Note that such a bigraph trivially exists. For 
example, one can take B = {V\ U U\, V% U U2, E) so that Vi, C/i, V2, U2 are disjoint sets of size t, every vertex 
of V\ is joined to every vertex of V2, and the edges between V\ and U2, as well as the edges between Ui and 
V2, form a matching of size t. Let Q' D Q be the set of graphs not containing B as an induced sub-bigraph. 
Then Q' is random-free by Theorem 13. 1\ as p b {B, W) = for every W £ Q! . 
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Let r = \V(B)\ and suppose that G = G(xi,X2, ■ ■ ■ ,x r , W) contains B as an induced sub-bigraph. Then 
there exists i so that x\, x%, ■ ■ ■ , x r € Si, as G is connected. It follows further that G is an induced subgraph 
of Hi, as no two vertices of G have the same neighborhood. Thus G contains no K t t subgraph, contradicting 
our assumption that G contains B. We conclude that supp(G(n, W)) C Q' for every positive integer n. 
By [LS061 Lemma 2.6] the sequence {G(n,W)} < ^' =1 converges to W with probability one. Thus W € Q! , as 
desired. 
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